Imaging device based on lens assembly with embedded filter

ABSTRACT

An imaging device for imaging of a local area surrounding the imaging device. The imaging device includes a lens assembly, a filtering element and a detector. The lens assembly is configured to receive light from a local area surrounding the imaging device and to direct at least a portion of the received light to the detector. The filtering element is placed in the imaging device within the lens assembly such that light is incident at a surface of the filtering element within a range of angles determined by a design range of angles at which the filtering element is designed to filter light. The detector is configured to capture image(s) of the local area including the filtered light. The imaging device can be integrated into a depth camera assembly for determining depth information of object(s) in the local area based on the captured image(s).

BACKGROUND

The present disclosure generally relates to an imaging device, andspecifically relates to an imaging device that includes a lens assemblywith an embedded filter.

An imaging device (camera or sensor), e.g., employed for depth sensingin augmented reality (AR) and virtual reality (VR) systems, typicallyutilizes a two-dimensional pixel array detector to measure and recordlight back-scattered from one or more objects in a scene. Other methodsfor depth sensing are based on a time-of-flight technique, whichmeasures a round trip travel time-of-light projected into the scene andreturning to pixels on a sensor array. In general, an imaging devicecaptures images of a scene based on light coming from one or moreobjects in the scene being detected by one or more pixels of a detectorincluded in the imaging device. The problem related to operation of animaging device is related to designing a compact and efficient camerathat can produce quality images in both indoor and outdoor environmentswhere background ambient light can strongly interfere with measurements.Thus, light received by the imaging device needs to be efficientlyfiltered within an assembly of the imaging device in order to blockundesired light components, e.g., background ambient light and/or lightof specific band(s).

SUMMARY

An imaging device is presented herein. The imaging device includes alens assembly, a filtering element and a detector. The lens assembly isconfigured to receive light from a local area surrounding the imagingdevice and to direct at least a portion of the received light to thedetector. The filtering element is placed in the imaging device withinthe lens assembly such that light is incident at a surface of thefiltering element within a range of angles, wherein the range of anglesis determined by a design range of angles at which the filtering elementis designed to filter light. The detector is configured to capture oneor more images of the local area including the filtered light. In someembodiments, the lens assembly generates collimated light using thereceived light, the collimated light composed of light rayssubstantially parallel to an optical axis. The surface of the filteringelement is perpendicular to the optical axis, and the collimated lightis incident on the surface of the filtering element. The filteringelement may be configured to reduce an intensity of a portion of thecollimated light to generate the filtered light.

A depth camera assembly (DCA) can integrate the imaging device. The DCAdetermines depth information associated with one or more objects in alocal area. The DCA further comprises a light generator and a controllercoupled to the light generator and the imaging device. The lightgenerator is configured to illuminate the local area with illuminationlight in accordance with emission instructions. The controller generatesthe emission instructions and provides the emission instructions to thelight generator. The controller further determines depth information forthe one or more objects based in part on the captured one or moreimages.

A head-mounted display (HMD) can further integrate the DCA with theimaging device. The HMD further includes an electronic display and anoptical assembly. The HMD may be, e.g., a virtual reality (VR) system,an augmented reality (AR) system, a mixed reality (MR) system, or somecombination thereof. The electronic display is configured to emit imagelight. The optical assembly is configured to direct the image light toan eye-box of the HMD corresponding to a location of a user's eye, theimage light comprising the depth information of the one or more objectsin the local area determined by the DCA.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example imaging device, in accordance with an embodiment.

FIG. 2 is a diagram of a head-mounted display (HMD), which may includethe imaging device in FIG. 1, in accordance with an embodiment.

FIG. 3 is a cross section of a front rigid body of the HMD in FIG. 2, inaccordance with an embodiment.

FIG. 4 is an example depth camera assembly (DCA), which may include theimaging device in FIG. 1, in accordance with an embodiment.

FIG. 5 is a block diagram of a HMD system in which a console operates,in accordance with an embodiment.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles, or benefits touted, of the disclosure described herein.

DETAILED DESCRIPTION

An imaging device (camera) presented herein includes a lens assemblywith a plurality of lens elements and a filtering element (e.g., aninterference filter). The filtering element is configured to pass aparticular band or bands of light received by the imaging device.However, often times, the filtering element may be of a type thatoperates with a designed passband for light incident within a particularrange of angles to a surface of the filtering element. For example, insome embodiments, the filtering element may operate with the designedpassband on light having incidence that is substantially perpendicularto a surface of the filtering element, e.g., ±5 degrees from normalincidence.

For a dichroic (interference) filtering element, a passband of thefiltering element (i.e., filter edges) may shift when the filteringelement operates on light having an angle of incidence (AOI) thatsubstantially deviates from a desired AOI for which the filteringelement is designed, e.g., when the AOI deviates by more than ±5 degreesfrom normal incidence. In this case, the filtering element would notpass desired wavelength(s) of received light as the passband of thefiltering element has been shifted to another band that typically doesnot overlap with a desired band. To avoid the filter edge shifting dueto incident light impinging on a surface of the filtering element awayfrom its designed AOI (e.g., normal incidence), the filtering element isplaced within the lens assembly of the imaging device where optical raysof the received light incident on the surface of the filtering elementare impinging substantially at the designed AOI of the filteringelement, e.g., within ±5 degrees from the designed AOI. The filteringelement embedded at a preferred location into the camera propagates theoptical rays in a particular desired band of light to generate filteredlight. The filtering element presented herein also blocks some or all ofthe optical rays of other light band(s).

In some embodiments, the camera is integrated into a depth cameraassembly (DCA). The DCA may further include an illumination source and acontroller. The illumination source illuminates a local area surroundingsome or all of the DCA with illumination light (e.g., structured light)in accordance with emission instructions from the controller. The cameracaptures one or more images of the local area including the filteredlight. The controller determines depth information for one or moreobjects in the local area based in part on the captured one or moreimages.

In some embodiments, the DCA with the camera is integrated into ahead-mounted display (HMD) that captures data describing depthinformation in the local area surrounding some or all of the HMD. TheHMD may be part of, e.g., a virtual reality (VR) system, an augmentedreality (AR) system, a mixed reality (MR) system, or some combinationthereof. The HMD further includes an electronic display and an opticalassembly. The electronic display is configured to emit image light. Theoptical assembly is configured to direct the image light to an eye-boxof the HMD corresponding to a location of a user's eye, the image lightcomprising the depth information of the one or more objects in the localarea determined by the DCA.

FIG. 1 is an imaging device 100, in accordance with an embodiment. Theimaging device 100 is configured for imaging of a local area 105surrounding some or all of the imaging device 100. The imaging device100 may be located in an indoor environment. Alternatively, the imagingdevice 100 may be located in an outdoor environment. The imaging device100 includes a lens assembly 110, a filtering element 115, a detector120, and a controller 125 coupled to the detector 120.

The lens assembly 110 receives light 130 coming from the local area 105.The lens assembly 110 includes a plurality of optical elements (e.g.,lenses) encased in a housing (not shown). The plurality of opticalelements are organized into at least a first lens subassembly 135 inoptical series with a second lens subassembly 140, with the filteringelement 115 there between. An air gap may exist between the first lenssubassembly 135 and the second lens subassembly 140. The first lenssubassembly 135 may include one or more optical elements (lenses,apertures, etc.) that generate light 145 using the received light 130.The light 145 generated by the first lens subassembly 135 is composed oflight rays that are diverge and/or converge no more than a thresholdvalue. The amount of divergence or convergence is based in part on adesign range of angles of the filtering element 115 discussed below. Insome embodiments, light rays of the light 145 are substantially parallelto an optical axis 150. The second lens subassembly 140 directs lighttoward the detector 120. The second lens subassembly 140 may include aprism (not shown in FIG. 1) or one or more other optical elements(lenses) that focus light toward a sensor area of the detector 120.

The filtering element 115 is configured to reduce an intensity of aportion of the light 145 to generate filtered light 155. In someembodiments, the filtering element 115 is implemented as an interferencefilter that passes a particular band of light. The filtering element 115is positioned within the lens assembly 110, e.g., between the first lenssubassembly 135 that generates the light 145 and the second lenssubassembly 140 that directs the filtered light 155 toward the detector120 as light 160.

The filtering element 115 is placed such that the light 145 is incidentat a surface 165 of the filtering element 115 within a range of angles.And the range of angles is determined by a design range of angles atwhich the filtering element 115 is designed to filter light. The designrange of angles for a typical infrared interference filter isapproximately ±5 degrees from an angle of incidence (AOI) for which thefiltering element 115 is designed. In the embodiment shown in FIG. 1,the AOI for which the filtering element 115 is designed is normalincidence. In this embodiment, the filtering element 115 is designed tofilter light for a design range of angles of the light 145 whose centerangle (i.e., angle between an axis perpendicular to a surface 165 of thefiltering element 115 and a center axis of a cone of light whose apex isat the surface 165) is substantially perpendicular to the surface 165.In alternate embodiments, the filtering element 115 may be designed tooperate at some other range of angles (e.g., 45±5 degrees, where 45degrees is the center angle and an angle between a lateral surface ofthe cone and the apex of the cone is 5 degrees) of the light incident onthe surface 165. And the filtering element 115 may be placed in thecamera 100 such that the light 145 is incident within the design rangeof angles. It should be noted—that as a size of the design range ofangles increases, the corresponding amount of collimation of the light145 may decrease. For example, if the design range of angles as a centerangle of 0 degrees and is ±0.1 degree, to have efficient operation ofthe filtering element 115 and avoid the filter edge shifting, the light145 should be substantially collimated and incident within ±0.1 degreeof normal to the surface 165. In contrast, if the design range of anglesis ±15 degrees (again with a center angle of 0 degrees), efficientoperation of the filtering element 115 without the filter edge shiftingmay be obtained with light that is not collimated and may beconverging/diverging—so long as the light 145 is incident on the surface165 at an angle within the design range of angles.

In FIG. 1, the light 145 has light rays substantially parallel to theoptical axis 150 and is incident on the surface 165 of the filteringelement. And the design range of angles of the filtering element 115 issuch that for efficient operation without the filter edge shiftingincident light should be at normal incidence. As the light 145 issubstantially collimated, the filtering element 115 can efficiently passa particular band of the light 145 and block undesired band(s) of thelight 145. In one embodiment, the filtering element 115 is configured topass a portion of the light 145 in a visible band. In anotherembodiment, the filtering element 115 is configured to pass a portion ofthe light 145 in an infrared band. In one embodiment, the surface 165 ofthe filtering element 115 is flat. In this case, the filtering element115 may not add/subtract any optical power to/from the incident light145. In another embodiment, the surface 165 is curved. In this case, thefiltering element 115 may be configured to add/subtract a definedoptical power to/from the incident light 145.

In some embodiments, the surface 165 of the filtering element 115 iscoated (e.g., with a metal or dichroic coating) to reflect a portion ofthe light 145 having one or more wavelengths outside a defined band. Thecoated surface 165 of the filtering element 115 also propagates otherportion of the light 145 having one or more other wavelengths within thedefined band to generate the filtered light 155.

The detector 120 captures one or more images of the local area 105including the filtered light 155 (i.e., the light 160 on a sensorsurface of the detector 120). In one embodiment, the detector 120 is aninfrared detector configured to capture images in an infrared band. Inanother embodiment, the detector 120 is configured to capture an imagelight of a visible band. The detector 120 may be implemented as acharge-coupled device (CCD) detector, a complementarymetal-oxide-semiconductor (CMOS) detector or some other types ofdetectors (not shown in FIG. 1). The detector 120 may be configured tooperate with a frame rate in the range of mHz to approximately 1 KHzwhen performing detection of objects in the local area 105. In someembodiments, the detector 120 is implemented as a two-dimensional pixelarray for capturing light signals related to the filtered light 155. Inother embodiments, the detector 120 is implemented as a single pixeldetector for capturing light signals related to the filtered light 155over a defined amount of time.

In some embodiments, the detector 120 is configured to capture the oneor more images of the local area 105 by capturing, at each pixel of thedetector 120, a light signal related to the filtered light 155 for eachtime instant of one or more time instants. A controller 165 coupled tothe detector 120 may be configured to determine depth information forone or more object in the local area 105 based on one or more lightsignals related to the filtered light 155 captured at each pixel of thedetector 120 during the one or more time instants.

In some embodiments, the received light 130 includes ambient light (notshown in FIG. 1) and a portion of illumination light (not shown inFIG. 1) reflected from one or more objects in the local area 105. Thefiltering element 115 may be configured to generate the filtered light155 substantially composed of the portion of illumination lightreflected from the one or more objects in the local area 105. Thedetector 120 may be configured to capture the portion of reflectedillumination light. The controller 165 may be configured to determinedepth information for the one or more objects in the local area 105based in part on the captured portion of the reflected illuminationlight.

In some embodiments, the imaging device 100 can be a component of a DCA,as disclosed in conjunction with FIGS. 3-4. In some embodiments, theimaging device 100 can be a component of a HMD, as disclosed inconjunction with FIGS. 2-3.

FIG. 2 is a diagram of a HMD 200, in accordance with an embodiment. TheHMD 200 may include the imaging device 100 (now shown in FIG. 2). TheHMD 200 may be part of, e.g., a VR system, an AR system, a MR system, orsome combination thereof. In embodiments that describe AR system and/ora MR system, portions of a front side 202 of the HMD 200 are at leastpartially transparent in the visible band (˜380 nm to 750 nm), andportions of the HMD 200 that are between the front side 202 of the HMD200 and an eye of the user are at least partially transparent (e.g., apartially transparent electronic display). The HMD 200 includes a frontrigid body 205, a band 210, and a reference point 215. The HMD 200 alsoincludes a DCA with the imaging device 100 configured to determine depthinformation of a local area surrounding some or all of the HMD 200. TheHMD 200 also includes an imaging aperture 220 and an illuminationaperture 225, and an illumination source of the DCA emits light (e.g.,structured light) through the illumination aperture 225. The imagingdevice 100 of the DCA captures light from the illumination source thatis reflected from the local area through the imaging aperture 220.

The front rigid body 205 includes one or more electronic displayelements (not shown in FIG. 2), one or more integrated eye trackingsystems (not shown in FIG. 2), an Inertial Measurement Unit (IMU) 230,one or more position sensors 235, and the reference point 215. In theembodiment shown by FIG. 2, the position sensors 235 are located withinthe IMU 230, and neither the IMU 230 nor the position sensors 235 arevisible to a user of the HMD 200. The IMU 230 is an electronic devicethat generates fast calibration data based on measurement signalsreceived from one or more of the position sensors 235. A position sensor235 generates one or more measurement signals in response to motion ofthe HMD 200. Examples of position sensors 235 include: one or moreaccelerometers, one or more gyroscopes, one or more magnetometers,another suitable type of sensor that detects motion, a type of sensorused for error correction of the IMU 230, or some combination thereof.The position sensors 235 may be located external to the IMU 230,internal to the IMU 230, or some combination thereof.

FIG. 3 is a cross section 300 of the front rigid body 205 of the HMD 200shown in FIG. 2. As shown in FIG. 3, the front rigid body 205 includesan electronic display 310 and an optical assembly 320 that togetherprovide image light to an eye-box 325. The eye-box 325 is a region inspace that is occupied by a user's eye 330. For purposes ofillustration, FIG. 3 shows a cross section 300 associated with a singleeye 330, but another optical assembly 320, separate from the opticalassembly 320, provides altered image light to another eye of the user.

The electronic display 310 generates image light. In some embodiments,the electronic display 310 includes an optical element that adjusts thefocus of the generated image light. The electronic display 310 displaysimages to the user in accordance with data received from a console (notshown in FIG. 3). In various embodiments, the electronic display 310 maycomprise a single electronic display or multiple electronic displays(e.g., a display for each eye of a user). Examples of the electronicdisplay 310 include: a liquid crystal display (LCD), an organic lightemitting diode (OLED) display, an inorganic light emitting diode (ILED)display, an active-matrix organic light-emitting diode (AMOLED) display,a transparent organic light emitting diode (TOLED) display, some otherdisplay, a projector, or some combination thereof. The electronicdisplay 310 may also include an aperture, a Fresnel lens, a convex lens,a concave lens, a diffractive element, a waveguide, a filter, apolarizer, a diffuser, a fiber taper, a reflective surface, a polarizingreflective surface, or any other suitable optical element that affectsthe image light emitted from the electronic display. In someembodiments, one or more of the display block optical elements may haveone or more coatings, such as anti-reflective coatings.

The optical assembly 320 magnifies received light from the electronicdisplay 310, corrects optical aberrations associated with the imagelight, and the corrected image light is presented to a user of the HMD200. At least one optical element of the optical assembly 320 may be anaperture, a Fresnel lens, a refractive lens, a reflective surface, adiffractive element, a waveguide, a filter, or any other suitableoptical element that affects the image light emitted from the electronicdisplay 310. Moreover, the optical assembly 320 may include combinationsof different optical elements. In some embodiments, one or more of theoptical elements in the optical assembly 320 may have one or morecoatings, such as anti-reflective coatings, dichroic coatings, etc.Magnification of the image light by the optical assembly 320 allowselements of the electronic display 310 to be physically smaller, weighless, and consume less power than larger displays. Additionally,magnification may increase a field-of-view of the displayed media. Forexample, the field-of-view of the displayed media is such that thedisplayed media is presented using almost all (e.g., 110 degreesdiagonal), and in some cases all, of the user's field-of-view. In someembodiments, the optical assembly 320 is designed so its effective focallength is larger than the spacing to the electronic display 310, whichmagnifies the image light projected by the electronic display 310.Additionally, in some embodiments, the amount of magnification may beadjusted by adding or removing optical elements.

As shown in FIG. 3, the front rigid body 105 further includes a DCA 340for determining depth information of one or more objects in a local area345 surrounding some or all of the HMD 200. The DCA 340 includes theimaging device 100 shown in FIG. 1. The DCA 340 further includes a lightgenerator 350, and a controller 360 that may be coupled to both thelight generator 350 and the imaging device 100. The light generator 350emits light through the illumination aperture 225. In accordance withembodiments of the present disclosure, the light generator 350 isconfigured to illuminate the local area 345 with light 365 in accordancewith emission instructions generated by the controller 360.

The light generator 350 may include a plurality of emitters that eachemits light having certain characteristics (e.g., wavelength,polarization, coherence, temporal behavior, etc.). The characteristicsmay be the same or different between emitters, and the emitters can beoperated simultaneously or individually. In one embodiment, theplurality of emitters could be, e.g., laser diodes (e.g., edgeemitters), inorganic or organic LEDs, a vertical-cavity surface-emittinglaser (VCSEL), or some other source. In some embodiments, a singleemitter or a plurality of emitters in the light generator 350 can emitone or more light beams. More details about the DCA 340 that includesthe light generator 350 are disclosed in conjunction with FIG. 4.

The imaging device 100 integrated into the DCA 340 may be configured tocapture, through the imaging aperture 220, at least a portion of thelight 365 reflected from the local area 345. The imaging device 100captures one or more images of one or more objects in the local area 345illuminated with the light 365. The controller 360 coupled to theimaging device 355 is also configured to determine depth information forthe one or more objects based on the captured portion of the reflectedlight. In some embodiments, the controller 360 provides the determineddepth information to a console (not shown in FIG. 3) and/or anappropriate module of the HMD 200 (e.g., a varifocal module, not shownin FIG. 3). The console and/or the HMD 200 may utilize the depthinformation to, e.g., generate content for presentation on theelectronic display 310.

In some embodiments, the front rigid body 205 further comprises an eyetracking system (not shown in FIG. 3) that determines eye trackinginformation for the user's eye 330. The determined eye trackinginformation may comprise information about an orientation of the user'seye 330 in an eye-box, i.e., information about an angle of an eye-gaze.An eye-box represents a three-dimensional volume at an output of a HMDin which the user's eye is located to receive image light. In oneembodiment, the user's eye 330 is illuminated with structured light.Then, the eye tracking system can use locations of the reflectedstructured light in a captured image to determine eye position andeye-gaze. In another embodiment, the eye tracking system determines eyeposition and eye-gaze based on magnitudes of image light captured over aplurality of time instants.

In some embodiments, the front rigid body 105 further comprises avarifocal module (not shown in FIG. 3). The varifocal module may adjustfocus of one or more images displayed on the electronic display 310,based on the eye tracking information. In one embodiment, the varifocalmodule adjusts focus of the displayed images and mitigatesvergence-accommodation conflict by adjusting a focal distance of theoptical assembly 320 based on the determined eye tracking information.In another embodiment, the varifocal module adjusts focus of thedisplayed images by performing foveated rendering of the one or moreimages based on the determined eye tracking information. In yet anotherembodiment, the varifocal module utilizes the depth information from thecontroller 360 to generate content for presentation on the electronicdisplay 310.

FIG. 4 is an example DCA 400, in accordance with an embodiment. The DCA400 is configured for depth sensing over a large field-of-view. The DCA400 includes the imaging device 100 shown in FIG. 1. The DCA 400 furtherincludes a light generator 405 and a controller 410 coupled to both thelight generator 405 and the imaging device 100. The DCA 400 may beconfigured to be a component of the HMD 200 in FIG. 2. Thus, the DCA 400may be an embodiment of the DCA 340 in FIG. 3, and the light generator405 may be an embodiment of the light generator 350 in FIG. 3.

The light generator 405 is configured to illuminate and scan a localarea 420 with illumination light in accordance with emissioninstructions from the controller 410. The light generator 405 mayinclude an illumination source 425, a diffractive optical element (DOE)430 and a projection assembly 435.

The illumination source 425 generates and directs light toward a portionof the DOE 430. The illumination source 425 includes a light emitter 440and a beam conditioning assembly 445. The light emitter 440 isconfigured to emit one or more optical beams 450, based in part on theemission instructions from the controller 410. In some embodiments, thelight emitter 440 includes an array of laser diodes that emit the one ormore optical beams 450 in an infrared band. In other embodiments, thelight emitter 440 includes an array of laser diodes that emit the one ormore optical beams 450 in a visible band. In some embodiments, the lightemitter 440 emits the one or more optical beams 450 as structured lightof a defined pattern (e.g., dot pattern, or line pattern). Alternativelyor additionally, the light emitter 440 emits the one or more opticalbeams 450 as temporally modulated light based in part on the emissioninstructions from the controller 410 to generate temporally modulatedillumination of the local area 420, e.g., in addition to structuredillumination.

The beam conditioning assembly 445 collects the one or more opticalbeams 450 emitted from the light emitter 440 and directs the one or moreoptical beams 450 toward at least a portion of the DOE 430. The beamconditioning assembly 445 is composed of one or more optical elements(lenses). In some embodiments, the beam conditioning assembly 445includes a collimation assembly and a prism (not shown in FIG. 4). Thecollimation assembly includes one or more optical elements (lenses) thatcollimate the one or more optical beams 450 into collimated light. Theprism is an optical element that directs the collimated light into theDOE 430. In alternate embodiments, the beam conditioning assembly 445includes a single hybrid optical element (lens) that both collimates theone or more optical beams 450 to generate collimated light and directsthe collimated light into the portion of the DOE 430.

The DOE 430 generates diffracted scanning beams 455 from the one or moreoptical beams 450, based in part on the emission instructions from thecontroller 410. The DOE 430 may be implemented as: an acousto-opticdevice, a liquid crystal on Silicon (LCOS) device, a spatial lightmodulator, a digital micro-mirror device, a microelectromechanical (MEM)device, some other diffraction grating element, or combination thereof.In some embodiments, operation of the DOE 430 can be controlled, e.g.,based in part on the emission instructions from the controller 410. Forexample, the controller 410 may control a voltage level applied to theDOE 430 or a radio frequency of a signal controlling a transducer of theDOE 430 (not shown in FIG. 4) to adjust a diffraction angle of the DOE430 for generating the diffracted scanning beams 455 with a variablespatial resolution and/or variable field-of-view. Having ability todynamically adjust a spatial resolution and/or a field-of-view of thediffracted scanning beams 455 (and of the illumination light 460)provides flexibility to scanning of different areas and various types ofobjects in the local area 420.

For a preferred diffraction efficiency, the DOE 430 may be configured todiffract the one or more optical beams 450 incident to at least aportion of the DOE 430 at an angle that satisfies the Bragg matchingcondition to form the diffracted scanning beams 455 based in part on theemission instructions from the controller 410. In some embodiments, theDOE 430 can be configured to generate the diffracted scanning beams 455as polarized light (e.g., circularly polarized light) by orienting theone or more optical beams 450 to, e.g., a liquid crystal in the DOE 430in a geometry satisfying the Bragg matching condition. Note that thediffracted scanning beams 450 can be either right handed circularlypolarized or left handed circularly polarized based on the liquidcrystal in the DOE 430. In some embodiments, a state of polarization(SOP) of the one or more optical beams 450 incident to the DOE 430matches an eigenstate of polarization at the Bragg angle for achievingmaximum diffraction efficiency of the DOE 430.

The projection assembly 435 is positioned in front of the DOE 430. Theprojection assembly 435 includes one or more optical elements (lenses).The projection assembly 435 projects the diffracted scanning beams 455as the illumination light 460 into the local area 420, e.g., over a widefield-of-view. In some embodiments, the illumination light 460 isstructured light of a defined pattern (e.g., dot pattern or linepattern). A pattern of the illumination light 460 may be dynamicallyadjustable over time based in part on the emission instructions from thecontroller 410 provided to the light emitter 440 and/or the DOE 430. Theillumination light 460 illuminates portions of the local area 420,including one or more objects in the local area 420. As the pattern ofthe illumination light 460 is dynamically adjustable over time,different portions of the local area 420 may be illuminated in differenttime instants. Reflected light 465 may be generated based on reflectionof the illumination light 465 from the one or more objects in the localarea 420.

The imaging device 100 integrated into the DCA 400 captures one or moreimages of the one or more objects in the local area 420 by capturing atleast a portion of the reflected light 465. The imaging device 100receives the reflected light 465 and background ambient light 470. Thebackground ambient light 470 is emitted from at least one ambient lightsource 475, e.g., sun in an outdoor environment or at least one lamp inan outdoor environment. In some embodiments, the background ambientlight 470 reaching the imaging device 100 may be also reflected from theone or more objects in the local area 420. The filtering element 115 ofthe imaging device 100 in FIG. 1 propagates a portion of the reflectedlight 465 within a particular band (e.g., visible band, or infraredband) to the detector 120 (not shown in FIG. 4) of the imaging device100. The filtering element 115 of the imaging device 100 is alsoconfigured to block or mitigate propagation of other portion of thereflected light 465 outside the particular band. The filtering element115 of the imaging device 100 may be further configured to block ormitigate propagation of the ambient light 475. Thus, the imaging device100 is configured to function as a band sensitive camera. Note that asdescribed above with regard to FIG. 1, the filtering element 115 ispositioned within a lens assembly of the camera 100 to receive lightwithin a design range of angles. And accordingly, is able to efficientlyfilter light received from the local area 420.

In some embodiments, the imaging device 100 may further include apolarizing element (not shown in FIG. 4) placed in front of the detector420 for receiving and propagating a portion of the reflected light 465of a particular polarization. The polarizing element can be a linearpolarizer, a circular polarizer, an elliptical polarizer, etc. Thepolarizing element can be implemented as a thin film polarizer(absorptive, reflective), a quarter wave plate combined with a linearpolarizer, etc. The reflected light 465 may be selected from a groupconsisting of linearly polarized light (vertical and horizontal), righthanded circularly polarized light, left handed circularly polarizedlight, and elliptically polarized light. Note that polarization of thereflected light 465 can be different than polarization of theillumination light 460 that illuminates the local area 420.

The controller 410 controls operations of various components of the DCA400 in FIG. 4. In some embodiments, the controller 410 provides emissioninstructions to the light emitter 440 to control intensity of theemitted one or more optical beams 450, modulation (spatial, frequency,temporal, etc.) of the one or more optical beams 450, a time durationduring which the light emitter 440 is activated, etc. The controller 410may further create the emission instructions for controlling operationsof the DOE 430 to dynamically adjust a pattern of the diffractedscanning beams 455 and a pattern the illumination light 460 thatilluminates the local area 420. In some embodiments, the controller 410can control, based in part on the emission instructions, operations ofthe DOE 430 such that intensity of the diffracted scanning beams 455varies over time. In other embodiments, the controller 410 can control,based in part on the emission instructions, operations of the DOE 430such that a phase of the diffracted scanning beams 455 varies over time.

In some embodiment, the DOE 430 is implemented as an acousto-opticdevice. The controller 410 may create the emission instructions whichinclude a radio frequency at which the DOE 430 is driven. The controller410 may generate the emission instructions based on, e.g., apredetermined list of values for the radio frequency stored in a look-uptable of the controller 410. In an embodiment, the predetermined radiofrequencies are stored as waveforms in an electronic chip, e.g., in adirect digital synthesizer (not shown in FIG. 4) coupled to thecontroller 410. In another embodiment, the emission instructions arecreated by a voice control integrated into the controller 410. Upon averbal request, the voice control of the controller 410 computes a radiofrequency for driving the DOE 430 to generate the diffracted scanningbeams 455 and the illumination light 460 of a specific spatial frequencysuitable for detection of stationary object(s) and/or tracking of movingobject(s) in the local area 420 at a certain distance from the imagingdevice 100.

The controller 410 can modify the radio frequency at which the DOE 430is driven to adjust a diffraction angle at which the one or more opticalbeams 450 are diffracted. In this way, the controller 410 can instructthe DOE 430 to scan a plurality of diffraction angles at which the oneor more optical beams 450 are diffracted and interfered to form thediffracted scanning beams 455 and the illumination light 460 of aparticular pattern. A radio frequency at which the DOE 430 is drivencontrols a separation of the optical beams 450 diffracted by the DOE430. Hence, a spatial frequency of the resulting diffracted scanningbeams 455 (and of the illumination light 460) directly depends on theradio frequency at which the DOE 430 is driven.

In some embodiments, the controller 410 controls a voltage level appliedto the DOE 430 implemented as a LC-based diffractive optical element todynamically vary a diffraction angle of the LC-based diffractive opticalelement to form the diffracted scanning beams 455 having a pattern thatvaries over time. In other embodiments, the controller 410 modifies overtime a spatial frequency of a modulation signal applied to the opticalbeams 450 via the DOE 430 implemented as a spatial light modulator todynamically adjust a pattern of the diffracted scanning beams 455 and apattern of the illumination light 360. In yet other embodiments, thecontroller 410 dynamically reconfigures subsets of micro-mirror cells inthe DOE 430. By reconfiguring, over a plurality of time instants,different subsets of the micro-mirror cells for absorption andreflection of incident light, the controller 410 dynamically adjusts apattern of the diffracted scanning beams 455 and a pattern of theillumination light 460.

As shown in FIG. 4, the controller 410 is further coupled to the imagingdevice 100 and can be configured to determine depth information for theone or more objects in the local area 420. The controller 410 determinesdepth information for the one or more objects based in part on the oneor more images captured by the imaging device 100. The controller 410may be integrated into the imaging device 100. In this case, thecontroller 410 may be an embodiment of the controller 165 in FIG. 1.

The controller 410 may be configured to determine the depth informationbased on phase-shifted patterns of light captured by the imaging device100 distorted by shapes of the one or more objects in the local area420, and to use triangulation calculation to obtain a depth map of thelocal area 420. Alternatively, the controller 410 may be configured todetermine the depth information based on time-of-flight information andinformation about a pattern of the reflected light 465 distorted byshapes of the one or more objects. In some embodiments, the controller410 can be configured to determine the depth information based onpolarization information of the reflected light 465 and/or polarizationinformation of the illumination light 460.

In some embodiments, the DCA 400 is configured as part of a HMD, e.g.,the HMD 200 in FIG. 2. In one embodiment, the DCA 400 provides thedetermined depth information to a console coupled to the HMD 200. Theconsole is then configured to generate content for presentation on anelectronic display of the HMD 200, based on the depth information. Inanother embodiment, the DCA 400 provides the determined depthinformation to a module of the HMD 200 that generates content forpresentation on the electronic display of the HMD 200, based on thedepth information. In an alternate embodiment, the DCA 400 is integratedinto a HMD 200 as part of an AR system. In this case, the DCA 400 may beconfigured to sense and display objects behind a head of a user wearingthe HMD 200 or display objects recorded previously. In yet otherembodiment, the DCA 400 is integrated into a base station or a sensorbar external to the HMD 200. In this case, the DCA 400 may be configuredto sense various body parts of a user wearing the HMD 200, e.g., theuser's lower body. In yet other embodiment, the DCA 400 is configured asa stand-alone VR capture device. In some other embodiments, the DCA 400is configured to determine eye tracking information (i.e., position andorientation) of one or both eyes of a user wearing the HMD 200implemented as part of a VR system, an AR system, or a MR system.

System Environment

FIG. 5 is a block diagram of one embodiment of a HMD system 500 in whicha console 510 operates. The HMD system 500 may operate in a VR systemenvironment, an AR system environment, a MR system environment, or somecombination thereof. The HMD system 500 shown by FIG. 5 comprises a HMD505 and an input/output (I/O) interface 515 that is coupled to theconsole 510. While FIG. 5 shows an example HMD system 500 including oneHMD 505 and on I/O interface 515, in other embodiments any number ofthese components may be included in the HMD system 500. For example,there may be multiple HMDs 505 each having an associated I/O interface515, with each HMD 505 and I/O interface 515 communicating with theconsole 510. In alternative configurations, different and/or additionalcomponents may be included in the HMD system 500. Additionally,functionality described in conjunction with one or more of thecomponents shown in FIG. 5 may be distributed among the components in adifferent manner than described in conjunction with FIG. 5 in someembodiments. For example, some or all of the functionality of theconsole 510 is provided by the HMD 505.

The HMD 505 is a head-mounted display that presents content to a usercomprising virtual and/or augmented views of a physical, real-worldenvironment with computer-generated elements (e.g., two-dimensional (2D)or three-dimensional (3D) images, 2D or 3D video, sound, etc.). In someembodiments, the presented content includes audio that is presented viaan external device (e.g., speakers and/or headphones) that receivesaudio information from the HMD 505, the console 510, or both, andpresents audio data based on the audio information. The HMD 505 maycomprise one or more rigid bodies, which may be rigidly or non-rigidlycoupled together. A rigid coupling between rigid bodies causes thecoupled rigid bodies to act as a single rigid entity. In contrast, anon-rigid coupling between rigid bodies allows the rigid bodies to moverelative to each other. An embodiment of the HMD 505 is the HMD 200described above in conjunction with FIG. 2.

The HMD 505 includes a DCA 520, an electronic display 525, an opticalassembly 530, one or more position sensors 535, an IMU 540, an optionaleye tracking system 545, and an optional varifocal module 550. Someembodiments of the HMD 505 have different components than thosedescribed in conjunction with FIG. 5. Additionally, the functionalityprovided by various components described in conjunction with FIG. 5 maybe differently distributed among the components of the HMD 505 in otherembodiments.

The DCA 520 captures data describing depth information of a local areasurrounding some or all of the HMD 505. The DCA 520 can compute thedepth information using the data (e.g., based on a captured portion of astructured light pattern), or the DCA 520 can send this information toanother device such as the console 510 that can determine the depthinformation using the data from the DCA 520.

The DCA 520 includes a light generator, an imaging device and acontroller. The light generator of the DCA 520 is configured toilluminate the local area with illumination light in accordance withemission instructions. The imaging device of the DCA 520 includes a lensassembly, a filtering element and a detector. The lens assembly isconfigured to receive light from a local area surrounding the imagingdevice and to direct at least a portion of the received light to thedetector. The filtering element may be placed in the imaging devicewithin the lens assembly such that light is incident at a surface of thefiltering element within a range of angles, wherein the range of anglesis determined by a design range of angles at which the filtering elementis designed to filter light. The detector is configured to capture oneor more images of the local area including the filtered light. In someembodiments, the lens assembly generates collimated light using thereceived light, the collimated light composed of light rayssubstantially parallel to an optical axis. The surface of the filteringelement is perpendicular to the optical axis, and the collimated lightis incident on the surface of the filtering element. The filteringelement may be configured to reduce an intensity of a portion of thecollimated light to generate the filtered light. The controller of theDCA 520 generates the emission instructions and provides the emissioninstructions to the light generator. The controller of the DCA 520further determines depth information for the one or more objects basedin part on the captured one or more images. The DCA 520 may be anembodiment of the DCA 340 in FIG. 3 or the DCA 400 in FIG. 4.

The electronic display 525 displays two-dimensional or three-dimensionalimages to the user in accordance with data received from the console510. In various embodiments, the electronic display 525 comprises asingle electronic display or multiple electronic displays (e.g., adisplay for each eye of a user). Examples of the electronic display 525include: a liquid crystal display (LCD), an organic light emitting diode(OLED) display, an inorganic light emitting diode (ILED) display, anactive-matrix organic light-emitting diode (AMOLED) display, atransparent organic light emitting diode (TOLED) display, some otherdisplay, or some combination thereof. In some embodiments, theelectronic display 525 may represent the electronic display 310 in FIG.3.

The optical assembly 530 magnifies image light received from theelectronic display 525, corrects optical errors associated with theimage light, and presents the corrected image light to a user of the HMD505. The optical assembly 530 includes a plurality of optical elements.Example optical elements included in the optical assembly 530 include:an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, areflecting surface, or any other suitable optical element that affectsimage light. Moreover, the optical assembly 530 may include combinationsof different optical elements. In some embodiments, one or more of theoptical elements in the optical assembly 530 may have one or morecoatings, such as partially reflective or anti-reflective coatings.

Magnification and focusing of the image light by the optical assembly530 allows the electronic display 525 to be physically smaller, weighless and consume less power than larger displays. Additionally,magnification may increase the field-of-view of the content presented bythe electronic display 525. For example, the field-of-view of thedisplayed content is such that the displayed content is presented usingalmost all (e.g., approximately 110 degrees diagonal), and in some casesall, of the user's field-of-view. Additionally in some embodiments, theamount of magnification may be adjusted by adding or removing opticalelements.

In some embodiments, the optical assembly 530 may be designed to correctone or more types of optical error. Examples of optical error includebarrel or pincushion distortions, longitudinal chromatic aberrations, ortransverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, chromatic aberrations or errorsdue to the lens field curvature, astigmatisms, or any other type ofoptical error. In some embodiments, content provided to the electronicdisplay 525 for display is pre-distorted, and the optical assembly 530corrects the distortion when it receives image light from the electronicdisplay 525 generated based on the content. In some embodiments, theoptical assembly 530 may represent the optical assembly 320 in FIG. 3.

The IMU 540 is an electronic device that generates data indicating aposition of the HMD 505 based on measurement signals received from oneor more of the position sensors 535 and from depth information receivedfrom the DCA 520. A position sensor 535 generates one or moremeasurement signals in response to motion of the HMD 505. Examples ofposition sensors 535 include: one or more accelerometers, one or moregyroscopes, one or more magnetometers, another suitable type of sensorthat detects motion, a type of sensor used for error correction of theIMU 540, or some combination thereof. The position sensors 535 may belocated external to the IMU 540, internal to the IMU 540, or somecombination thereof.

Based on the one or more measurement signals from one or more positionsensors 535, the IMU 540 generates data indicating an estimated currentposition of the HMD 505 relative to an initial position of the HMD 505.For example, the position sensors 535 include multiple accelerometers tomeasure translational motion (forward/back, up/down, left/right) andmultiple gyroscopes to measure rotational motion (e.g., pitch, yaw,roll). In some embodiments, the position sensors 535 may represent theposition sensors 235 in FIG. 2. In some embodiments, the IMU 540 rapidlysamples the measurement signals and calculates the estimated currentposition of the HMD 505 from the sampled data. For example, the IMU 540integrates the measurement signals received from the accelerometers overtime to estimate a velocity vector and integrates the velocity vectorover time to determine an estimated current position of a referencepoint on the HMD 505. Alternatively, the IMU 540 provides the sampledmeasurement signals to the console 510, which interprets the data toreduce error. The reference point is a point that may be used todescribe the position of the HMD 505. The reference point may generallybe defined as a point in space or a position related to the HMD's 505orientation and position.

The IMU 540 receives one or more parameters from the console 510. Theone or more parameters are used to maintain tracking of the HMD 505.Based on a received parameter, the IMU 540 may adjust one or more IMUparameters (e.g., sample rate). In some embodiments, certain parameterscause the IMU 540 to update an initial position of the reference pointso it corresponds to a next position of the reference point. Updatingthe initial position of the reference point as the next calibratedposition of the reference point helps reduce accumulated errorassociated with the current position estimated the IMU 540. Theaccumulated error, also referred to as drift error, causes the estimatedposition of the reference point to “drift” away from the actual positionof the reference point over time. In some embodiments of the HMD 505,the IMU 540 may be a dedicated hardware component. In other embodiments,the IMU 540 may be a software component implemented in one or moreprocessors. In some embodiments, the IMU 540 may represent the IMU 230in FIG. 2.

In some embodiments, the eye tracking system 545 is integrated into theHMD 505. The eye tracking system 545 determines eye tracking informationassociated with an eye of a user wearing the HMD 505. The eye trackinginformation determined by the eye tracking system 545 may compriseinformation about an orientation of the user's eye, i.e., informationabout an angle of an eye-gaze. In some embodiments, the eye trackingsystem 545 is integrated into the optical assembly 530. An embodiment ofthe eye-tracking system 545 may comprise an illumination source and animaging device (camera).

In some embodiments, the varifocal module 550 is further integrated intothe HMD 505. The varifocal module 550 may be coupled to the eye trackingsystem 545 to obtain eye tracking information determined by the eyetracking system 545. The varifocal module 550 may be configured toadjust focus of one or more images displayed on the electronic display525, based on the determined eye tracking information obtained from theeye tracking system 545. In this way, the varifocal module 550 canmitigate vergence-accommodation conflict in relation to image light. Thevarifocal module 550 can be interfaced (e.g., either mechanically orelectrically) with at least one of the electronic display 525 and atleast one optical element of the optical assembly 530. Then, thevarifocal module 550 may be configured to adjust focus of the one ormore images displayed on the electronic display 525 by adjustingposition of at least one of the electronic display 525 and the at leastone optical element of the optical assembly 530, based on the determinedeye tracking information obtained from the eye tracking system 545. Byadjusting the position, the varifocal module 550 varies focus of imagelight output from the electronic display 525 towards the user's eye. Thevarifocal module 550 may be also configured to adjust resolution of theimages displayed on the electronic display 525 by performing foveatedrendering of the displayed images, based at least in part on thedetermined eye tracking information obtained from the eye trackingsystem 545. In this case, the varifocal module 550 provides appropriateimage signals to the electronic display 525. The varifocal module 550provides image signals with a maximum pixel density for the electronicdisplay 525 only in a foveal region of the user's eye-gaze, whileproviding image signals with lower pixel densities in other regions ofthe electronic display 525. In one embodiment, the varifocal module 550may utilize the depth information obtained by the DCA 520 to, e.g.,generate content for presentation on the electronic display 525.

The I/O interface 515 is a device that allows a user to send actionrequests and receive responses from the console 510. An action requestis a request to perform a particular action. For example, an actionrequest may be an instruction to start or end capture of image or videodata or an instruction to perform a particular action within anapplication. The I/O interface 515 may include one or more inputdevices. Example input devices include: a keyboard, a mouse, a gamecontroller, or any other suitable device for receiving action requestsand communicating the action requests to the console 510. An actionrequest received by the I/O interface 515 is communicated to the console510, which performs an action corresponding to the action request. Insome embodiments, the I/O interface 515 includes an IMU 540 thatcaptures calibration data indicating an estimated position of the I/Ointerface 515 relative to an initial position of the I/O interface 515.In some embodiments, the I/O interface 515 may provide haptic feedbackto the user in accordance with instructions received from the console510. For example, haptic feedback is provided when an action request isreceived, or the console 510 communicates instructions to the I/Ointerface 515 causing the I/O interface 515 to generate haptic feedbackwhen the console 510 performs an action.

The console 510 provides content to the HMD 505 for processing inaccordance with information received from one or more of: the DCA 520,the HMD 505, and the I/O interface 515. In the example shown in FIG. 5,the console 510 includes an application store 555, a tracking module560, and an engine 565. Some embodiments of the console 510 havedifferent modules or components than those described in conjunction withFIG. 5. Similarly, the functions further described below may bedistributed among components of the console 510 in a different mannerthan described in conjunction with FIG. 5.

The application store 555 stores one or more applications for executionby the console 510. An application is a group of instructions, that whenexecuted by a processor, generates content for presentation to the user.Content generated by an application may be in response to inputsreceived from the user via movement of the HMD 505 or the I/O interface515. Examples of applications include: gaming applications, conferencingapplications, video playback applications, or other suitableapplications.

The tracking module 560 calibrates the HMD system 500 using one or morecalibration parameters and may adjust one or more calibration parametersto reduce error in determination of the position of the HMD 505 or ofthe I/O interface 515. For example, the tracking module 560 communicatesa calibration parameter to the DCA 520 to adjust the focus of the DCA520 to more accurately determine positions of structured light elementscaptured by the DCA 520. Calibration performed by the tracking module560 also accounts for information received from the IMU 540 in the HMD505 and/or an IMU 540 included in the I/O interface 515. Additionally,if tracking of the HMD 505 is lost (e.g., the DCA 520 loses line ofsight of at least a threshold number of structured light elements), thetracking module 560 may re-calibrate some or all of the HMD system 500.

The tracking module 560 tracks movements of the HMD 505 or of the I/Ointerface 515 using information from the DCA 520, the one or moreposition sensors 535, the IMU 540 or some combination thereof. Forexample, the tracking module 550 determines a position of a referencepoint of the HMD 505 in a mapping of a local area based on informationfrom the HMD 505. The tracking module 560 may also determine positionsof the reference point of the HMD 505 or a reference point of the I/Ointerface 515 using data indicating a position of the HMD 505 from theIMU 540 or using data indicating a position of the I/O interface 515from an IMU 540 included in the I/O interface 515, respectively.Additionally, in some embodiments, the tracking module 560 may useportions of data indicating a position or the HMD 505 from the IMU 540as well as representations of the local area from the DCA 520 to predicta future location of the HMD 505. The tracking module 560 provides theestimated or predicted future position of the HMD 505 or the I/Ointerface 515 to the engine 555.

The engine 565 generates a 3D mapping of the area surrounding some orall of the HMD 505 (i.e., the “local area”) based on informationreceived from the HMD 505. In some embodiments, the engine 565determines depth information for the 3D mapping of the local area basedon information received from the DCA 520 that is relevant for techniquesused in computing depth. The engine 565 may calculate depth informationusing one or more techniques in computing depth from structured light.In various embodiments, the engine 565 uses the depth information to,e.g., update a model of the local area, and generate content based inpart on the updated model.

The engine 565 also executes applications within the HMD system 500 andreceives position information, acceleration information, velocityinformation, predicted future positions, or some combination thereof, ofthe HMD 505 from the tracking module 560. Based on the receivedinformation, the engine 565 determines content to provide to the HMD 505for presentation to the user. For example, if the received informationindicates that the user has looked to the left, the engine 565 generatescontent for the HMD 505 that mirrors the user's movement in a virtualenvironment or in an environment augmenting the local area withadditional content. Additionally, the engine 565 performs an actionwithin an application executing on the console 510 in response to anaction request received from the I/O interface 515 and provides feedbackto the user that the action was performed. The provided feedback may bevisual or audible feedback via the HMD 505 or haptic feedback via theI/O interface 515.

In some embodiments, based on the eye tracking information (e.g.,orientation of the user's eye) received from the eye tracking system545, the engine 565 determines resolution of the content provided to theHMD 505 for presentation to the user on the electronic display 525. Theengine 565 provides the content to the HMD 505 having a maximum pixelresolution on the electronic display 525 in a foveal region of theuser's gaze, whereas the engine 565 provides a lower pixel resolution inother regions of the electronic display 525, thus achieving less powerconsumption at the HMD 505 and saving computing cycles of the console510 without compromising a visual experience of the user. In someembodiments, the engine 565 can further use the eye tracking informationto adjust where objects are displayed on the electronic display 525 toprevent vergence-accommodation conflict.

Additional Configuration Information

The foregoing description of the embodiments of the disclosure has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of thedisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the disclosure may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that isproduced by a computing process described herein. Such a product maycomprise information resulting from a computing process, where theinformation is stored on a non-transitory, tangible computer readablestorage medium and may include any embodiment of a computer programproduct or other data combination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the disclosure be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thedisclosure, which is set forth in the following claims.

What is claimed is:
 1. An imaging device comprising: a lens assemblycomprising a first subassembly and a second subassembly, the firstsubassembly configured to: receive light from a local area surroundingthe imaging device; and generate collimated light using the receivedlight, the collimated light including reflected illumination light andambient light; a filtering element positioned in an air gap between thefirst subassembly and the second subassembly, the filtering elementincluding a surface that is perpendicular to the optical axis, thefiltering element configured to reduce an intensity of a portion of thecollimated light including the ambient light to generate filtered lightsubstantially without the ambient light, light rays of the collimatedlight generated by the first subassembly are incident within a range ofangles around a center angle of incidence to the surface of thefiltering element for which the filtering element is designed to reducean intensity of incident light; and a detector configured to capture oneor more images of the local area including the filtered light directedto the detector by the second subassembly.
 2. The imaging device ofclaim 1, wherein the filtering element is further configured to: blockthe portion of the collimated light having one or more wavelengthsoutside a defined band; and propagate other portion of the collimatedlight having one or more other wavelengths within the defined band togenerate the filtered light.
 3. The imaging device of claim 1, whereinthe detector is further configured to: capture the one or more images bycapturing, at each pixel of the detector, a light signal related to thefiltered light for each time instant of one or more time instants. 4.The imaging device of claim 3, further comprising: a controller coupledto the detector configured to determine depth information for the localarea based on one or more light signals related to the filtered lightcaptured at each pixel of the detector during the one or more timeinstants.
 5. The imaging device of claim 1, wherein: the received lightincludes the ambient light and the reflected illumination lightreflected from one or more objects in the local area; the filteringelement is configured to generate the filtered light substantiallycomposed of the reflected illumination light; and the detector isconfigured to capture the reflected illumination light.
 6. The imagingdevice of claim 5, further comprising: a controller coupled to thedetector configured to determine depth information for the one or moreobjects based in part on the reflected illumination light.
 7. Theimaging device of claim 1, wherein the imaging device is a component ofa depth camera assembly.
 8. A depth camera assembly (DCA) comprising: alight generator configured to illuminate a local area with illuminationlight in accordance with emission instructions; a lens assemblycomprising a first subassembly and a second subassembly, the firstsubassembly configured to: receive light from the local area, thereceived light including ambient light and a portion of the illuminationlight reflected from one or more objects in the local area, and generatecollimated light using the received light, the collimated lightincluding the reflected illumination light and the ambient light; afiltering element positioned in an air gap between the first subassemblyand the second subassembly, the filtering element including a surfacethat is perpendicular to the optical axis, the filtering elementconfigured to reduce an intensity of a portion of the collimated lightincluding the ambient light to generate filtered light substantiallywithout the ambient light, light rays of the collimated light generatedby the first subassembly are incident within a range of angles around acenter angle of incidence to the surface of the filtering element forwhich the filtering element is designed to reduce an intensity ofincident light; a detector configured to capture one or more images ofthe local area including the filtered light directed to the detector bythe second subassembly; and a controller configured to: generate theemission instructions, provide the emission instructions to the lightgenerator, and determine depth information for the one or more objectsbased in part on the captured one or more images.
 9. The DCA of claim 8,wherein the filtering element is further configured to: block theportion of the collimated light having one or more wavelengths outside adefined band; and propagate other portion of the collimated light havingone or more other wavelengths within the defined band to generate thefiltered light.
 10. The DCA of claim 8, wherein the detector is furtherconfigured to: capture the one or more images by capturing, at eachpixel of the detector, a light signal related to the filtered light foreach time instant of one or more time instants.
 11. The DCA of claim 10,wherein the controller is further configured to: determine depthinformation for the local area based on one or more light signalsrelated to the filtered light captured at each pixel of the detectorduring the one or more time instants.
 12. The DCA of claim 8, whereinthe light generator includes: an illumination source configured to emitone or more optical beams; a diffractive optical element (DOE) thatgenerates, based in part on the emission instructions, diffractedscanning beams from the one or more optical beams; and a projectionassembly configured to project the diffracted scanning beams as theillumination light into the local area.
 13. The DCA of claim 12, whereinthe illumination source includes: a light emitter configured to emit theone or more optical beams, based in part on the emission instructions; acollimation assembly configured to collimate the one or more opticalbeams into collimated beams; and a prism configured to direct thecollimated beams into the DOE.
 14. The DCA of claim 12, wherein theillumination source includes: a light emitter configured to emit the oneor more optical beams, based in part on the emission instructions; and asingle optical element configured to: collimate the one or more opticalbeams to generate collimated beams, and direct the collimated beams intothe DOE.
 15. The DCA of claim 12, wherein the one or more optical beamsare temporally modulated providing the illumination light to betemporally modulated.
 16. The DCA of claim 8, wherein the illuminationlight includes structured light of a defined pattern.
 17. The DCA ofclaim 8, wherein the DCA is a component of a head-mounted display.
 18. Ahead-mounted display (HMD) comprising: a display configured to emitimage light; a light generator configured to illuminate a local areawith illumination light in accordance with emission instructions; a lensassembly comprising a first subassembly and a second subassembly, thefirst subassembly configured to: receive light from the local area, thereceived light including ambient light and a portion of the illuminationlight reflected from one or more objects in the local area, and generatecollimated light using the received light, the collimated lightincluding the reflected illumination light and the ambient light; afiltering element positioned in an air gap between the first subassemblyand the second subassembly, the filtering element including a surfacethat is perpendicular to the optical axis, the filtering elementconfigured to reduce an intensity of a portion of the collimated lightincluding the ambient light to generate filtered light substantiallywithout the ambient light, light rays of the collimated light generatedby the first subassembly are incident within a range of angles around acenter angle of incidence to the surface of the filtering element forwhich the filtering element is designed to reduce an intensity ofincident light; a detector configured to capture one or more images ofthe local area including the filtered light directed to the detector bythe second subassembly; a controller configured to: generate theemission instructions, provide the emission instructions to the lightgenerator, and determine depth information for the one or more objectsbased in part on the captured one or more images; and an opticalassembly configured to direct the image light to an eye-box of the HMDcorresponding to a location of a user's eye, the image light comprisingthe determined depth information.